Image brightness controlling apparatus and method thereof

ABSTRACT

Provided are an image brightness control device and an image brightness control method for improving the definition of brightness of the entire image and/or improving the definition of brightness of local areas using local brightness information. The image brightness controlling device includes: a preprocessing unit acquiring an offset table for controlling a dynamic range corresponding to an image range of an input image using brightness values of color data of the input image; and a tone mapping unit mapping the offset table onto the color data. It is possible to improve the definition of brightness so as to correspond to the characteristic of the input image, by automatically considering how to reflect a distribution characteristic of an image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims foreign priority benefits under 35 U.S.C. .sctn.119(a)-(d) to PCT/KR08/002524, filed May 6, 2008, which is herebyincorporated by reference in its entirety.

BACKGROUND

1. Technical Field

The present invention relates to an image brightness controlling device,and more particularly, to an image brightness controlling device and amethod thereof, which improves the definition of brightness by the useof the definition of brightness of an entire image and/or locallyimproves the definition of brightness by the use of local brightnessinformation.

2. Description of the Related Art

Conventional methods of controlling a dynamic range of an input imageare described as follows. In the following description, it is assumedthat the input image has an n-bit resolution (where n is a naturalnumber).

(1) Contrast Enhancement

A dynamic range is controlled between 0 and 2^(n)-1 by applying theminimum pixel value and the maximum pixel value in an image toExpression 1.Output=(2^(n)−1)×(Input−Min)/(Max−Min)  Expression 1

Here, Output is a pixel value of an output image, Input is a pixel valueof an input image, Max is the maximum pixel value of the input image,and Min is the minimum pixel value of the input image.

(2) Histogram Equalization

FIG. 1 is a transformation function graph of an input image and anoutput image to which a histogram equalization method is applied.

First, a cumulative density function (CDF) of pixel values in an imageis calculated using histogram information of the image. Then, the scaleof the cumulative density function is controlled between 0 and 2^(n)-1and the cumulative density function is used as a transformation function100 of the input image and the output image.

When the number of specific pixel values is great, the slope of thetransformation function 100 rapidly varies thereby enhancing the dynamicrange of the corresponding pixel values.

In the contrast enhancement method or the histogram equalization method,the dynamic range is always controlled into values between 0 and2^(n)-1, regardless of the range of pixel values of the input image.

However, when the range of pixel values of the input image is verynarrow but the dynamic range is forcibly enhanced into the valuesbetween 0 and 2^(n)-1, the quality of image is not improved, but noisesrather increases and the image is made to seem very unnatural.

In recent years, electronic apparatuses requiring an image processingdevice such as a digital camera, a camcorder, or a camera mounted on amobile phone were widely used. When an image having high contrast istaken with the image processing device, an image biased to a dark sideor a bright side is obtained. That is, when the dynamic range of animage is great, an image different from an image seen with a naked eyeis obtained by the image processing device. Here, the dynamic range ofan image means a range of brightness between the darkest part and thebrightest part of the image. For example, when a person standing by abright window is taken with a camera, an image of a dark person standingby the bright window is obtained which has been distinguished with anaked eye. An image improving algorithm such as a retinex algorithm isused to solve the above-mentioned problem.

An image brightness controlling method using a multi-scale retinex withcolor restoration (MSRCR) employs Expression 2.

$\begin{matrix}{{{R_{i}\left( {x_{1},x_{2}} \right)} = {{a_{i}\left( {x_{1},x_{2}} \right)}{\sum\limits_{k = 1}^{K}\;{W_{k}\left( {{\log\;{I_{i}\left( {x_{1},x_{2}} \right)}} - {\log\left\lbrack {{F_{k}\left( {x_{1},x_{2}} \right)} \times {I_{i}\left( {x_{1},x_{2}} \right)}} \right\rbrack}} \right)}}}}{{i = 1},\ldots\mspace{14mu},S}{{{F_{k}\left( {x_{1},x_{2}} \right)} = {\kappa\;{\exp\left\lbrack {{- \left( {x_{1}^{2} + x_{2}^{2}} \right)}/\sigma_{k}^{2}} \right\rbrack}}},{\kappa = {1/\left( {\sum\limits_{x_{1}}\;{\sum\limits_{x_{2}}\;{F\left( {x_{1},x_{2}} \right)}}} \right)}}}{{\alpha_{i}\left( {x_{1},x_{2}} \right)} = {\log\left( {{{SI}_{i}\left( {x_{1},x_{2}} \right)}/{\sum\limits_{n = 1}^{S}\;{I_{n}\left( {x_{1},x_{2}} \right)}}} \right)}}} & {{Expression}\mspace{14mu} 2}\end{matrix}$

Here, i represents an order of a spectral band, S represents a grayimage when it is “1” and represents a color image when it is “3”, (x₁,x₂) represents a coordinate of an image, I represents an input image, Rrepresents an output image having been subjected to the MSRCR process,F_(k) represents a k-th Gaussian surround function, σ_(k) represents astandard deviation of the k-th Gaussian surround function, K representsthe number of surround functions, W_(k) a weight (generally 1/K)associated with F_(k), and α_(i) represents a color restorationcoefficient in an i-th spectral band.

Since the image brightness controlling method using Expression 2 isperformed in a frequency domain, there is a problem with troublesomecalculation such as Fourier transformation should be performed. Inaddition, plural frame memories for storing input images and Gaussianfunctions are required and an additional memory for storing a table forlog calculation is also required. Since the calculation could not beperformed in real time, there is also a problem in that the method couldbe performed only as a post process.

SUMMARY

Accordingly, a goal of the invention is to provide an image brightnesscontrolling device and a method thereof, in which the contrastenhancement method and the histogram equalization method are properlycombined depending on a range of pixel values of an input image on thebasis of a characteristic that the contrast enhancement method isadvantageous when the range of pixel values of the input image is narrowand the histogram equalization method using a distributioncharacteristic of pixel values is advantageous when the range of pixelvalues of the input image is wide.

Another goal of the invention is to provide an image brightnesscontrolling device and a method thereof, which controls a dynamic rangeto be applied to an output image depending on the range of pixel valuesof an input image and improves the definition of brightness byautomatically considering how a distribution characteristic of an imageshould be reflected.

Another goal of the invention is to provide an image brightnesscontrolling device and a method thereof, which can improve thedefinition of brightness by controlling a degree of stretching in theunit of local regions of an input image.

Another goal of the invention is to provide an image brightnesscontrolling device and a method thereof, which can easily acquire thenatural quality of image with small noise by controlling inputparameters (variance value and asymmetric parameters) in considerationof a degree of noise of an input image.

Another goal of the invention is to provide an image brightnesscontrolling device and a method, which has an optimized memory using theminimum memory space without requiring troublesome calculations such asFourier transform and several pieces of frame memories.

Another goal of the invention is to provide an image brightnesscontrolling device and a method thereof, which is applicable in realtime, by performing a tone mapping operation in the unit of pixels of acurrent frame of an image using information acquired from a previousframe.

According to an aspect of the invention, there is provided an imagebrightness controlling device for controlling the definition ofbrightness of an output image depending on a brightness characteristicof an input image.

An image brightness controlling device according to an embodiment of theinvention may include: a preprocessing unit acquiring an offset tablefor controlling a dynamic range corresponding to an image range of aninput image using brightness values of color data of the input image;and a tone mapping unit mapping the offset table onto the color data.

The preprocessing unit may include: a dynamic range informationacquiring unit setting a predetermined region of the brightness valuesof pixels of the input image as the image range; a stretching unitstretching the brightness values of the input image by combining theimage range and a brightness control strength; and an offset calculatingunit calculating ratios of the brightness values in the input image andcalculating offset values of the brightness values from the ratios.

The dynamic range information acquiring unit may include: a histogramacquiring unit expressing the number of cases where the brightnessvalues appear in the pixels of the input image as a histogram; acumulative density function (CDF) calculating unit calculating acumulative density function obtained by accumulating the histogram withrespect to the brightness values; and an image range setting unitsetting a predetermined area of the cumulative density function to theimage range.

The image range setting unit may set the brightness values of thepixels, in which the values obtained by dividing the values of thecumulative density function of the pixels by the size of the input imagecorrespond to a boundary predetermined or input by a user, as a globalminimum value (global_min) and a global maximum value (global_max). Thestretching unit may determine a dynamic range to be applied to an outputimage by combining the global minimum value and the global maximum valueand the brightness control strength predetermined or input by the user.

The offset calculating unit may calculate a ratio at which a currentbrightness value occupies the input image from the value of thecumulative density function of the brightness values of the input image,and calculates the offset value from the ratio. The offset calculatingunit may calculate the offset values of all the pixels of the inputimage to acquire the offset table.

The color data may be HSV data and the preprocessing unit may controlthe dynamic range using the value of V of the HSV data.

The color data may be YUV data or YCbCr data and the preprocessing unitmay control the dynamic range using the value of Y of the color data.

The preprocessing unit may acquire local brightness averages of localareas obtained by dividing the input image into local areas having apredetermined size, and the tone mapping unit may perform a local tonemapping operation by comparing a bias applied value using a bias curvewith a predicted average of a pixel in the input image predicted fromthe local brightness values.

An image brightness control device according to another embodiment ofthe invention may include: a preprocessing unit acquiring dynamic rangeinformation corresponding to an image range of an input image usingbrightness values of color data of the input image and acquiring localbrightness averages of local areas obtained by dividing the input imageinto the local areas having a predetermined size; and a tone mappingunit performing a local tone mapping operation by comparing a biasapplied value using a bias curve with a predicted average of a pixel inthe input image predicted from the local brightness values.

The tone mapping unit may acquire the bias applied value of a pixel inthe input image, calculate the predicted average from the localbrightness averages, and stretch the brightness value of the pixel.

The preprocessing unit may include: a dynamic range informationacquiring unit setting a predetermined region of the brightness valuesof the pixels of the input image as an image range; and a localbrightness average calculating unit dividing the input image to aplurality of local areas and then calculating the local brightnessaverages which are averages of the brightness values of the pixels inthe plurality of local areas.

The dynamic range information acquiring unit may include: a histogramacquiring unit expressing the number of cases where the brightnessvalues appear in the pixels of the input image as a histogram; acumulative density function (CDF) calculating unit calculating acumulative density function obtained by accumulating the histogram withrespect to the brightness values; and an image range setting unitsetting a predetermined area of the cumulative density function to theimage range.

The tone mapping unit may include: a bias curve applying unit acquiringthe bias applied value by applying the brightness value of the pixel tothe bias curve, wherein the bias applied value is less than thebrightness value when the brightness value of the pixel is equal to orgreater than a threshold value and is greater than the brightness valuewhen the brightness value is less than the threshold value; a predictedaverage calculating unit calculating the predicted average from thelocal brightness averages by the use of a bi-linear interpolationmethod; and a stretching unit stretching the brightness value using adifference between the bias applied value and the predicted average.

The stretching unit may apply a weight predetermined or input from auser to the difference between the bias applied value and the predictedaverage. The weight may be set to enhance the degree of stretching asthe dynamic range based on the dynamic range information is wider. Theweight may be independently controlled on the basis of an asymmetryparameter input from the user.

According to another aspect of the invention, there is provided an imagebrightness controlling method of controlling the definition ofbrightness of an output image depending on a brightness characteristicof an input image and a recording medium having a program for executingthe method.

An image brightness controlling method according to an embodiment of theinvention may include the steps of: acquiring an offset table forcontrolling a dynamic range corresponding to an image range of an inputimage using brightness values of color data of the input image; andmapping the offset table onto the color data.

The step of acquiring the offset table may include the steps of: settinga predetermined region of the brightness values of pixels of the inputimage as the image range; stretching the brightness values of the inputimage by combining the image range and a brightness control strength.

The step of setting a predetermined area of the brightness values of thepixels in the input image may include the steps of: acquiring ahistogram from the number of cases where the brightness values appear inthe pixels of the input image as a histogram; calculating a cumulativedensity function obtained by accumulating the histogram with respect tothe brightness values; and setting a predetermined area of thecumulative density function to the image range.

The image brightness controlling method may further include the stepsof: after the step of stretching the brightness values of the inputimage by combining the image range and the brightness control strength,calculating ratios of the brightness values in the input image from thecumulative density function; and calculating offset values of thebrightness values from the ratios.

The step of setting the image range may include setting the brightnessvalues of the pixels, in which the values obtained by dividing thevalues of the cumulative density function of the pixels by the size ofthe input image correspond to a boundary predetermined or input by auser, as a global minimum value and a global maximum value.

The step of stretching may include determining a dynamic range to beapplied to an output image by combining the global minimum value and theglobal maximum value and the brightness control strength predeterminedor input by the user.

The step of calculating the offset may include calculating a ratio atwhich a current brightness value occupies the input image from the valueof the cumulative density function of the brightness values of the inputimage, and calculates the offset value from the ratio.

The step of calculating the offset may include calculating the offsetvalues of all the pixels of the input image to acquire the offset table.

The color data may be HSV data and the step of calculating the offsettable may include controlling the dynamic range using the value of V ofthe HSV data.

Alternatively, the color data may be YUV data or YCbCr data and the stepof calculating the offset table may include controlling the dynamicrange using the value of Y of the color data.

The image brightness control method may further include the steps of:acquiring local brightness averages of local areas obtained by dividingthe input image into local areas having a predetermined size, andperforming a local tone mapping operation by comparing a bias appliedvalue using a bias curve with a predicted average of a pixel in theinput image predicted from the local brightness values.

An image brightness control method according to another aspect of theinvention may include the steps of: acquiring dynamic range informationcorresponding to an image range of an input image using brightnessvalues of color data of the input image; acquiring local brightnessaverages of local areas obtained by dividing the input image into thelocal areas having a predetermined size; and performing a local tonemapping operation by comparing a bias applied value using a bias curvewith a predicted average of a pixel in the input image predicted fromthe local brightness values.

The step of acquiring the dynamic range information may include thesteps of: expressing the number of cases where the brightness valuesappear in the pixels of the input image as a histogram; calculating acumulative density function (CDF) obtained by accumulating the histogramwith respect to the brightness values; and setting a predetermined areaof the cumulative density function to the image range.

The step of calculating the local brightness averages may include thesteps of: dividing the input image to a plurality of local areas; andcalculating the local brightness averages which are averages of thebrightness values of the pixels in the plurality of local areas.

The step of performing the local tone mapping operation may include thesteps of: acquiring the bias applied value by applying the brightnessvalue of the pixel to the bias curve, wherein the bias applied value isless than the brightness value when the brightness value of the pixel isequal to or greater than a threshold value and is greater than thebrightness value when the brightness value is less than the thresholdvalue; calculating the predicted average from the local brightnessaverages by the use of a bi-linear interpolation method; and stretchingthe brightness value using a difference between the bias applied valueand the predicted average.

The step of stretching the brightness values may include applying aweight predetermined or input from a user to the difference between thebias applied value and the predicted average.

The weight may be set to enhance the degree of stretching as the dynamicrange based on the dynamic range information is wider. The weight may beindependently controlled on the basis of an asymmetry parameter inputfrom the user.

The other aspects, features, and advantages of the invention will becomeapparent from the accompanying drawings, the appended claims, and thedetailed description of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a graph illustrating a transformation function of an inputimage and an output image to which the histogram equalization methodshould be applied.

FIG. 2 is a diagram schematically illustrating a configuration of animaging apparatus according to an embodiment of the invention.

FIG. 3 is a diagram schematically illustrating an image brightnesscontrolling device according to a first embodiment of the invention.

FIG. 4 is a diagram schematically illustrating a configuration of apreprocessing unit according to the first embodiment of the invention.

FIG. 5 is a flowchart illustrating an image brightness controllingmethod of the preprocessing unit according to the first embodiment ofthe invention.

FIG. 6 is a diagram schematically illustrating an image brightnesscontrolling device according to a second embodiment of the invention.

FIG. 7 is a diagram schematically illustrating a dynamic rangeinformation acquiring unit.

FIG. 8 is a diagram illustrating a method of calculating a localbrightness average.

FIG. 9 is a diagram illustrating a bias curve graph.

FIG. 10 is a diagram illustrating a bi-linear interpolation method.

FIG. 11 is a diagram illustrating a weighting function graph.

FIG. 12 is a flowchart illustrating a method of controlling the imagebrightness in the unit of local regions.

FIG. 13 is a diagram illustrating an example of an original image.

FIG. 14 is a diagram illustrating an output image acquired by applyingthe image brightness controlling method according to the firstembodiment of the invention to the original image shown in FIG. 13.

FIG. 15 is a diagram illustrating another example of an original image.

FIG. 16 is a diagram illustrating an output image acquired by applyingthe image brightness controlling method according to the secondembodiment of the invention to the original image shown in FIG. 15.

DETAILED DESCRIPTION

The above-mentioned goals, features, and advantages of the inventionwill be apparent from the following detailed description with referenceto the accompanying drawings.

The invention can be variously modified in various embodiments andspecific embodiments will be described and shown in the drawings. Theinvention is not limited to the embodiments, but it should be understoodthat the invention includes all the modifications, equivalents, andreplacements belonging to the spirit and the technical scope of theinvention. When it is determined that detailed description of knowntechniques associated with the invention makes the gist of the inventionobscure, the detailed description will be omitted.

Terms, “first”, “second”, and the like, can be used to describe variouselements, but the elements are not limited to the terms. The terms areused only to distinguish one element from another element. For example,without departing from the scope of the invention, a first element maybe named a second element and the second element may be named the firstelement similarly. The term, “and/or”, includes a combination of pluralelements or any one of the plural elements.

If it is mentioned that an element is “connected to” or “coupled to”another element, it should be understood that the element may beconnected or coupled directly to another element or that still anotherelement may be interposed therebetween. On the contrary, if it ismentioned that an element is “connected directly to” or “coupleddirectly to” another element, it should be understood that still anotherelement is not interposed therebetween.

The terms used in the following description are used to merely describespecific embodiment, but are not intended to limit the invention. Anexpression of the singular number includes an expression of the pluralnumber, so long as it is clearly read differently. The terms such as“include”, “have”, and the like are intended to indicate that features,numbers, steps, operations, elements, components, or combinationsthereof used in the following description exist and that the possibilityof existence or addition of one or more different features, numbers,steps, operations, elements, components, or combinations thereof is notexcluded.

So long as they are not defined differently, all the terms used therein,which include technical or scientific terms, have the same meanings asgenerally understood by those skilled in the art. It should be analyzedthat the terms defined in dictionaries used in general have the samemeaning as in the contexts of the related art, but the terms should notbe analyzed ideal or excessively formal.

Hereinafter, exemplary embodiments of the invention will be described indetail with reference to the accompanying drawings. Like orcorresponding elements are denoted by like reference numerals regardlessof the drawing number and repeated description thereof is omitted.

FIG. 2 is a diagram schematically illustrating a configuration of animaging apparatus according to an embodiment of the invention.

An imaging apparatus 200 according to an embodiment of the inventionincludes an RGB-HSV transformation unit 210, an image brightnesscontrolling device 220, and an HSV-RGB transformation unit 230. Animaging apparatus 200 according to another embodiment of the inventionincludes an RGB-YUV transformation unit 215, an image brightnesscontrolling device 220, and a YUV-RGB transformation unit 235.

The imaging apparatus 200 according to the embodiment transforms RGBdata into HSV data of the color data, performs an image brightnesscontrolling operation, and then transforms the HSV data into the RGBdata again.

The RGB-HSV transformation unit 210 transforms the RGB data of an inputimage into the HSV data. In general, the input image has the RGB data asimage information of pixels. In order to control the image brightnessaccording to this embodiment, it is necessary to treat the imageinformation in a HSV color model.

In an RGB color model, any color is expressed by a compound colorcomponent of three colors of red (R), green (G), and blue (B) as thethree primary colors. The principle of additional mixture is applied toa color, and when three values of R, G, and B are 0, a black color isobtained.

In a CMY color model, three colors of cyan (C), magenta (M), and yellow(Y) are used as the primary colors and any color is expressed by acompound color component of the three primary colors. In this case, theprinciple of subtractive mixture is applied to a color, and when threevalues of C, M, and Y are 0, a white color is obtained.

The HSV color model is a model relatively matched with a human feelingon colors. Six colors of R, Y, G, C, B, and M are used as the primarycolors, are arranged in a circular circumference at an interval of 60degrees, and colors obtained by dividing them at an equivalent intervalare made by means of combination of the adjacent primary colors, therebycompleting the entire circumference, which is called a hue ring. Aposition in the hue ring is an H value of a hue. The saturation S hasthe greater value as it goes to the edge in a CIE Chromaticity diagram.When the value is 0, a center white color is obtained. The value Vrepresents the brightness of a color.

The RGB-HSV transformation unit 210 transforms pixel values (R, G, and Bvalues) of the input image corresponding to the RGB color model intopixel values (H, S, and V values) corresponding to the HSV color model.

The value of H indicates a hue, that is, the kind of colors (colorcomponents such as red, yellow, and blue), and expresses all theexpressible colors in the range of 0 to 360. In general, it isnormalized in the range of 0 to 100%. The value of S indicatessaturation, that is, the definition of colors, and the smaller value ofS means that it is closer to an achromatic color. The value of Vindicates a value, that is, the brightness of colors, in the range of 0to 100%. The smaller value of V means that it is darker and the greatervalue of V means that it is brighter.

The RGB-HSV transformation unit 210 performs the RGB-HSV transformationoperation using Expression 3.

$\begin{matrix}{{H = \begin{Bmatrix}{{{60 \times \frac{G - B}{{MAX} - {MIN}}} + 0},} & {{{if}\mspace{14mu}{MAX}} = {{R\mspace{14mu}{and}\mspace{14mu} G} \geq B}} \\{{{60 \times \frac{G - B}{{MAX} - {MIN}}} + 360},} & {{{if}\mspace{14mu}{MAX}} = {{R\mspace{14mu}{and}\mspace{14mu} G} < B}} \\{{{60 \times \frac{B - R}{{MAX} - {MIN}}} + 120},} & {{{if}\mspace{14mu}{MAX}} = G} \\{{{60 \times \frac{R - G}{{MAX} - {MIN}}} + 240},} & {{{if}\mspace{14mu}{MAX}} = B}\end{Bmatrix}}{S = \frac{{MAX} - {MIN}}{MAX}}{V = {MAX}}} & {{Expression}\mspace{14mu} 3}\end{matrix}$

Here, MAX represents the maximum value of the R, G, and B values of eachpixel of the input image and MIN represents the minimum value of the R,G, and B values of each pixel of the input image. H has a value in therange of 0 to 360 and S and V vary between 0 and 1.

When MAX and MIN are equal to each other (MAX=MIN, that is, S=0), H isnot defined. When the value of S is 0, it means that it is an achromaticcolor and does not have any color component. When MAX is 0, S is notdefined. When the value of V, it means that it has pure black and doesnot have any color component and any saturation component.

It is possible to acquire the values of H, S, and V of each pixel of theinput image from Expression 3.

The image brightness control device 220 controls the definition ofbrightness of the whole image using the value of V of the values of H,S, and V of each pixel of the input image acquired by the RGB-HSVtransformation unit 210.

The HSV-RGB transformation unit 230 transforms the values of H, S, and Vof the image, the brightness value (V) of which has been controlled bythe image brightness control device 220, into the values of R, G, and B.This corresponds to the inversion of the transformation performed by theRGB-HSV transformation unit 210.

An example of the HSV-RGB transformation is as follows.

(1) When the value of S is 0, the color is achromatic and R, G, and Bcorrespond to the value of V.

(2) When the value of S is not 0, Expression 4 is applied.

$\begin{matrix}{{H_{i} = {\left\lfloor \frac{H}{60} \right\rfloor\mspace{14mu}{mod}\mspace{14mu} 6}}{f = {\frac{H}{60} - H_{i}}}{p = {V\left( {1 - S} \right)}}{q = {V\left( {1 - {fS}} \right)}}{t = {V\left( {1 - {\left( {1 - f} \right)S}} \right)}}{{{{if}\mspace{14mu} H_{i}} = {\left. 0\rightarrow R \right. = V}},{G = t},{B = p}}{{{{if}\mspace{14mu} H_{i}} = {\left. 1\rightarrow R \right. = q}},{G = V},{B = p}}{{{{if}\mspace{14mu} H_{i}} = {\left. 2\rightarrow R \right. = p}},{G = V},{B = t}}{{{{if}\mspace{14mu} H_{i}} = {\left. 3\rightarrow R \right. = p}},{G = q},{B = V}}{{{{if}\mspace{14mu} H_{i}} = {\left. 4\rightarrow R \right. = t}},{G = p},{B = V}}{{{{if}\mspace{14mu} H_{i}} = {\left. 5\rightarrow R \right. = V}},{G = p},{B = q}}} & {{Expression}\mspace{14mu} 4}\end{matrix}$

Here, mod N is a function indicating the remainder of division by N.

The imaging apparatus 200 transforms the input image as data in the RGBcolor model into an output image in the same color model.

However, an image brightness control device according to an embodimentof the invention uses the value of V which is the brightness value ofthe HSV data in the HSV color model to control the image brightness.

An image brightness control device 220 according to another embodimentof the invention uses the value of Y which is the brightness value ofthe YUV data in the YUV color model to control the image brightness.

The YUV color model is a color model using the fact that a human eye isthe most sensitive to the intensity of light, where Y indicates theintensity of light, that is, luminance, and U and V indicate thechrominance.

An imaging apparatus 200 according to another embodiment of theinvention includes an RGB-YUV transformation unit 215 and a YUV-RGBtransformation unit 235.

The RGB-YUV transformation unit 215 transforms the pixel values (thevalues of R, G, and B) of the input image in the RGB color model intothe pixel values (the values of Y, U, and V) in the YUV color model bythe use of Expression 5.Y=0.299×R+0.587×G+0.114×BU=−0.1687×R−0.3313×G+0.5×B+128V=0.5×R−0.4187×G−0.0813×B+128   Expression 5

The image brightness control device 220 controls the image brightnessusing the value of Y acquired from Expression 5 and a method describedlater.

The YUV-RGB transformation unit 235 transforms the values of Y, U, and Vof each pixel of the image, the brightness value (the value of Y) hasbeen controlled by the image brightness control device 220, into thevalues of R, G, and B. This operation corresponds to the inversion ofthe transformation performed by the RGB-YUV transformation unit 215 anduses Expression 6.R=Y+1.370705×(V−128)G=Y−0.337633×(U−128)−0.698001×(V−128)B=Y+1.732446×(U−128)   Expression 6

An image brightness control device 220 according to another embodimentof the invention uses the value of Y which is the brightness value ofYCbCr data in a YCbCr color model to control the image brightness.

That is, it should be understood by those skilled in the art that theimage brightness control device 220 according to the invention controlsthe image brightness using the brightness value included in the colordata and the invention is applicable to color data including thebrightness value.

The HSV data of the color data will be mainly described now for thepurpose of convenient understanding and description. The configurationand function of the image brightness control device 220 controlling theimage brightness using the brightness value (the value of V) will bedescribed in detail.

The image brightness control device 220 controls the image brightnessusing the brightness value of the input image. According to a firstembodiment, an image brightness control device 220 can improve thedefinition of brightness of the whole image.

According to a second embodiment, an image brightness control device 220can improve the definition of brightness of local areas of the imageusing the local brightness information. According to a third embodiment,an image brightness control device 220 can primarily improve thedefinition of brightness of the whole image and secondarily improve thedefinition of brightness of the local areas using the local brightnessinformation.

Now, the image brightness control device 220 and a method for improvingthe definition of brightness of the whole image according to the firstembodiment will be described.

FIG. 3 is a diagram schematically illustrating a configuration of theimage brightness control device according to the first embodiment of theinvention, FIG. 4 is a diagram schematically illustrating aconfiguration of the preprocessing unit according to the firstembodiment of the invention, and FIG. 5 is a flow diagram illustrating amethod of controlling the image brightness in the preprocessing unitaccording to the first embodiment of the invention.

The image brightness control device 220 according to the firstembodiment of the invention includes a preprocessing unit 310 and a tonemapping unit 320 (see FIG. 3). The preprocessing unit 310 acquires anoffset table from the input image and the tone mapping unit 320 maps theacquired offset table onto the input image.

Here, the preprocessing unit 310 acquires the offset table from the N-thframe of the input image and the tone mapping unit 320 can map theacquired offset table onto the N-th frame or the subsequent frames (suchas the (N+1)-th frame and the (N+2)-th frame) of the input image.

Referring to FIG. 4, the preprocessing unit 310 includes a dynamic rangeinformation acquiring unit setting a predetermined area of thebrightness values of the pixels in the input image to an image range, astretching unit 317, and an offset calculating unit 319. Here, thedynamic range information acquiring unit includes a histogram acquiringunit 311, a CDF calculating unit 313, and an image range setting unit315.

Now, it is assumed that the input image has an n-bit (where n is anatural number) resolution. The histogram acquiring unit 311 scales thebrightness value V of the values of H, S, and V of each pixel of theinput image in a range of 0 to 2^(n)-1 and expresses the number of caseswhere the correspond value of V appears in the pixels in the input imageby the use of a histogram.

The CDF calculating unit 313 calculates a cumulative density function(hereinafter, referred to as “CDF”) in which the histogram acquired bythe histogram acquiring unit 311 is accumulated from the brightnessvalue of 0. When the brightness value is 2^(n)-1, the maximum CDF valueis equal to the size of the input image, that is, (the number ofhorizontal pixels)×(the number of vertical pixels), and means the totalnumber of pixels of the input image.

Since the maximum CDF value is the size of the input image, that is, thetotal number of pixels of the input image, the normalized CDF(normed_CDF) obtained by normalizing the maximum CDF value depending onthe brightness value may be further calculated. That is, when thebrightness value is 2^(n)-1, the maximum value can be normalized to be2^(n)-1.

The image range setting unit 315 sets the brightness values of thepixels, in which the values obtained by dividing the CDF values of thepixels by the size of the input image correspond to a boundarypredetermined or input by a user, as a global minimum value (global_min)and a global maximum value (global_max) so as to acquire a distributionrange of brightness of the pixels in the input image. When the valueobtained by dividing the CDF value by the size of the input imagecorresponds to A (for example, 0.001), the brightness value can be setas the global minimum value. When the value obtained by dividing the CDFvalue by the size of the input image corresponds to B (for example,0.999), the brightness value can be set as the global maximum value.Here, A and B are a real number between 0 and 1 and B is greater than A.

The selection in the range between the global minimum value and theglobal maximum value of the brightness values of the pixels in the inputimage is performed to exclude the influence on the noise. The brightnessvalues of which the ratio in the image is almost zero, (for example,salt and pepper noise) are neglected to perform the image processaccording to the embodiment of the invention.

The stretching unit 317 stretches the brightness value of the inputimage by combining the global minimum value and the global maximum valueset by the image range setting unit 315 and the brightness controlstrength as an input parameter predetermined or input from a user.

That is, the stretching unit acquires a modified minimum value(modi_min) which is a modified value of the global minimum value and amodified maximum value (modi_max) which is a modified value of theglobal maximum value and performs a stretching operation so as to locatethe brightness values of the pixels of the input image, which have beenlocated in the global minimum value and the global maximum value, in themodified minimum value and the modified maximum value. The modifiedminimum value and the modified maximum value determine the dynamic rangeto be applied to the output image.

The offset calculating unit 319 calculates a ratio at which the currentbrightness value occupies the input image from the CDF of the brightnessvalues of the input image and calculates the offset value from theratio. The offset calculating unit adds the calculated offset value tothe brightness value of the image stretched by the stretching unit 317to acquire an offset table of the brightness values of 0 to 2^(n)-1.Here, by considering the reflection degree of the distributioncharacteristic of the input image which can be acquired from the CDF, itis possible to improve the definition of brightness in accordance withthe characteristic of the input image.

The tone mapping unit 320 performs a tone mapping operation on thebrightness values of the pixels by applying the offset table acquired bythe preprocessing unit 310 to the input image. It is possible to improvethe definition of brightness of the tone-mapped output image so as tocorrespond to the brightness distribution characteristic of the inputimage.

An image brightness controlling method of the image brightness controldevice 220 will be described in detail now with reference to FIG. 5.FIG. 5 is a flow diagram illustrating the image brightness controllingmethod according to an embodiment of the invention.

TABLE 1 Terms Description image_size Size information of image, that is,horizontal(width) × vertical(height). Histogram Brightness values V(0~1)are scaled to 0~2^(n) − 1 and the number of cases where the valuesappear in image. CDF Function of histograms accumulated from 0 (maximumvalue is image_size when brightness value is 2^(n) − 1). normed_CDFMaximum CDF value is size of image, that is, horizontal × vertical. CDFobtained by normalizing it in the range of 0 to 2^(n) − 1. StrengthInput parameter determining brightness range of image and/or brightnesscontrol strength. Stretching Process of widening brightness range ofimage using input parameter strength global_min Value of CDF/(width ×height) corresponding to A (for example, 0.001). global_max Value ofCDF/(width × height) corresponding to B (for example, 0.999). modi_minModified value of global-min (stretching result) modi_max Modified valueof global-max (stretching result) offset_table Table of offsets to beadded to brightness values (0~255) drange_val Data indicating dynamicrange of original image

Table 1 shows meanings of terms used in the following description. Theimage brightness controlling method will be described with reference toTable 1.

The values of H, S, and V of the input image are acquired by the RGB-HSVconversion. The value of V, that is, the brightness value, is adjusted.Here, it is assumed that the brightness value of the input image has ann-bit (where n is a natural number) resolution.

The brightness values of the pixels in the input image are in the rangeof 0 to 1. The brightness values are scaled to the range of 0 to 2^(n)-1and the number of cases in which the values appear in the input image isexpressed, thereby acquiring the histogram (step S510).

Then, the cumulative density function in which the acquired histogram isaccumulated in the magnitude order of the brightness values (from 0 to2^(n)-1) is acquired (step S520). Here, when the brightness value is2^(n)-1, the maximum CDF value means the size of the input image, thatis, (the number of horizontal pixels (width))×(the number of verticalpixels (height)), which is the total number of pixels in the inputimage.

The image range of the pixels in the input image is set (step S530). Inorder to find out the distribution range of the brightness values of theinput image, the brightness values of the pixels, in which the valuesobtained by dividing the CDF values of the pixels by the size of theinput image correspond to the boundary predetermined or input by a user,are set as the global minimum value (global_min) and the global maximumvalue (global_max).

When the value obtained by dividing the CDF value by the size of theinput image corresponds to A (for example, 0.001), the brightness valuecan be set as the global minimum value. When the value obtained bydividing the CDF value by the size of the input image corresponds to B(for example, 0.999), the brightness value can be set as the globalmaximum value. Here, A and B are a real number between 0 and 1 and B isgreater than A.

In addition, a 7-tap method can be used to smooth the CDF curve. The7-tap method is a method using the average of the brightness values ofthe current pixel and the left three pixels and the right three pixelsabout the current pixel. The CDF curve can be further smoothed by themethod.

Since the maximum CDF value is the size of the input image, that is, thetotal number of pixels in the input image, the normalized CDF(normed_CDF) obtained by normalizing the maximum CDF value to correspondto the brightness value may be further calculated. That is, when thebrightness value is 2^(n)-1, the maximum value can be normalized to be2^(n)-1.

The brightness values of the input image are stretched (step S540) bycombining the global minimum value and the global maximum value set bythe image range setting unit 315 and the brightness control strength asan input parameter predetermined or input from a user. That is, amodified minimum value (modi_min) which is a modified value of theglobal minimum value and a modified maximum value (modi_max) which is amodified value of the global maximum value are acquired and thestretching operation is performed so as to locate the brightness valuesof the pixels of the input image, which have been located in the globalminimum value and the global maximum value, in the modified minimumvalue and the modified maximum value. The modified minimum value(modi_min) and the modified maximum value (modi_max) determine thedynamic range to be applied to the output image.

For example, the modified minimum value and the modified maximum valuecan be obtained using Expression 7.modi_min=global_min−(global_min−0)×strengthmodi_max=global_max+((2^(n)-1)−global_max)×strength   Expression 7

Here, the brightness control strength is predetermined or input as aninput parameter by a user. The brightness control strength is aparameter for determining the brightness range of an image and/or thecontrol strength of the brightness values.

Then, the offset table is acquired (step S550).

At this time, a first linear curve for stretching the input image isfirst acquired (step S552). A second linear curve is acquired from thenormalized CDF of the global minimum value and the global maximum value(step S554).

Examples of the first linear curve and the second linear curve areexpressed by Expression 8.y=gain1×x+offset1 (a first linear curve)y=gain2×x+offset2 (a second linear curve)gain1=(modi_max−modi_min)/(global_max−global_min)offset1=−gain1×global_min+modi_mingain2=(Normed_CDF (1, global_max+1)−Normed_CDF(1,global_min+1))/(global_max−global_min)offset2=−gain2×global_min+Normed_CDF (1,global_min+1))   Expression 8

Here, x represents the brightness value of the input image, y representsthe stretched brightness value, gain1 and gain2 represent the slopes ofthe first and second linear curves, offset1 and offset2 representintercepts of the first and second linear curves. The normed_CDF(1,a)represents the CDF value corresponding to (1,a) in the matrix of 1×2″.

The offset table of the brightness values of 1 to 2^(n)-1 is generatedusing gain1, gain2, offset1, and offset2 acquired from Expression 8(step S556).

When the brightness value of a pixel is smaller than the global minimumvalue, the brightness value of the pixel is set as the global minimumvalue. When the brightness value of a pixel is greater than the globalmaximum value, the brightness value of the pixel is set as the globalmaximum value (see Expression 9).

Expression 9 for i = 0:2^(n)−1 if i<global_min   value = global_min;else if i>global_max   value = global_max; else   value = i ;

Here, the brightness value (stretched_pixel) stretched by the use of thefirst linear curve of Expression 8 and the offset table is acquired bythe use of Expression 10.offset_table(1,i+1)=stretched_pixel+(Normed_CDF(1,value+1)−(gain2×value+offset2))×alpha  Expression 10

Here, offset_table(1,i+1) represents an offset value of the offset tablecorresponding to (1,i+1) in the matrixe of 1×2^(n) and alpha is acquiredfrom Expression 11.alpha=strength×0.4×(1−(value−30)²/(2^(n)-1)²)   Expression 11

By considering the reflection degree of the distribution characteristicof the input image which can be acquired from the CDF, that is, alpha,it is possible to improve the definition of brightness to correspond tothe characteristic of the input image.

By applying the acquired offset table to the input image, the brightnessvalues of the pixels are tone-mapped (step S560). The tone-mapped outputimage has the improved definition of brightness to correspond to thedistribution characteristic of the input image.

Hitherto, the image brightness control device and the image brightnesscontrol method applied to the entire image have been described.

Hereinafter, an image brightness control device and an image brightnesscontrol method for improving the definition of brightness local imagesusing local brightness information will be described.

FIG. 6 is a diagram schematically illustrating an image brightnesscontrol device according to a second embodiment of the invention andFIG. 7 is a diagram schematically illustrating a dynamic rangeinformation acquiring unit. FIG. 8 is a diagram illustrating a method ofcalculating a local brightness average, FIG. 9 is a diagram illustratinga graph of a bias curve, FIG. 10 is a diagram illustrating a bi-linearinterpolation method, and FIG. 11 is a diagram illustrating a graph of aweighting function.

The image brightness control device 220 according to the secondembodiment of the invention includes a preprocessing unit 610 and a tonemapping unit 620 (see FIG. 6). The preprocessing unit 610 acquiresdynamic range information and local brightness averages from the inputimage. The tone mapping unit 620 tone-maps the input image using theacquired dynamic range information and the acquired local brightnessaverages.

Here, the preprocessing unit 610 acquires the dynamic range informationand the local brightness averages of the N-th frame of the input image.The tone mapping unit 620 tone-maps the N-th frame or the subsequentframe ((N+1)-th frame, (N+2)-th frame, and the like) using the acquireddynamic range information and the local brightness averages.

The preprocessing unit 610 includes a dynamic range informationacquiring unit 612 and a local brightness average calculating unit 614.

The dynamic range information acquiring unit 612 acquires the dynamicrange information of the input image. The dynamic range informationacquiring unit acquires a histogram of the brightness values of thepixels in the input image, sets an image range having a predeterminedsize from the CDF values of the brightness values, and sets the imagerange as the dynamic range.

The local brightness average calculating unit 614 partitions the inputimage into local areas having a predetermined size and calculates thebrightness averages of the local areas. Thereafter, the tone mappingunit 620 performs a tone mapping operation using the local brightnessaverages.

The meanings of the terms used in the preprocessing unit 610 are shownin Table 2.

TABLE 2 Terms Description image_size Size information of image, that is,horizontal(width) × vertical(height). histogram Brightness values V(0~1)are scaled to 0~2^(n) − 1 and the number of cases where the valuesappear in image. CDF Function of histograms accumulated from 0 (maximumvalue is image_size when brightness value is 2^(n) − 1). global_minValue of CDF/(width × height) corresponding to A (for example, 0.001).global_max Value of CDF/(width × height) corresponding to B (forexample, 0.999). drange_val Data indicating dynamic range of originalimage mask_s Horizontal and vertical sizes of local areas mean_tableTable storing average brightness value of local areas (mask_s × mask_s).

The dynamic range information acquiring unit 612 includes a histogramacquiring unit 710, a CDF calculating unit 720, and an image rangesetting unit 730 (see FIG. 7). It is assumed that the input image has ann-bit (where n is a natural number) resolution.

The histogram acquiring unit 710 scales the brightness value V of thevalues of H, S, and V of the pixels of the input image in the range of 0to 2^(n)-1 and expresses the number of cases where the values of Vappear in the pixels in the input image as a histogram.

The CDF calculating unit 720 calculates the CDF in which the histogramacquired by the histogram acquiring unit 710 is accumulated from thebrightness value of 0. When the brightness value is 2^(n)-1, the maximumCDF value is the size of the input image, that is, (the number ofhorizontal pixels)×(the number of vertical pixels), and means the totalnumber of pixels (image_size) of the input image.

The image range setting unit 730 sets the brightness values of thepixels, in which the values obtained by dividing the CDF values of thepixels by the size of the input image correspond to a boundarypredetermined or input by a user, as a global minimum value (global_min)and a global maximum value (global_max) so as to acquire a distributionrange of brightness of the pixels in the input image. When the valueobtained by dividing the CDF value by the size of the input imagecorresponds to A (for example, 0.001), the brightness value can be setas the global minimum value. When the value obtained by dividing the CDFvalue by the size of the input image corresponds to B (for example,0.999), the brightness value can be set as the global maximum value.Here, A and B are a real number between 0 and 1 and B is greater than A.

The selection in the range between the global minimum value and theglobal maximum value of the brightness values of the pixels in the inputimage is performed to exclude the influence on the noise. The brightnessvalues of which the ratio in the image is almost zero, (for example,salt and pepper noise) are neglected to perform the image processaccording to the embodiment of the invention.

The dynamic range information of the input image is obtained from theglobal maximum value and the global minimum value set by the image rangesetting unit 730. That is, the value (global_max−global_min) obtained bysubtracting the global minimum value from the global maximum value isdetermined as the dynamic range.

The local brightness average calculating procedure of the localbrightness average calculating unit 614 is shown in FIG. 8. An inputimage 810 has the number of pixels corresponding to (the number ofhorizontal pixels (width))×(the number of vertical pixels (height). Theinput image 810 is divided into local areas 811, 812, 813, and 184having a predetermined size and the average of the brightness values ofeach local area 811, 812, 813, or 814 is calculated as the localbrightness average of the corresponding local area. Here, the localareas 811, 812, 813, and 814 having a predetermined size may be a squarearea having the same horizontal and vertical size as mask₁₃ s.

The local brightness averages are stored as an average table 820 in amemory unit (such as a memory, RAM, or ROM) in the imaging apparatus 100or an external memory unit.

The tone mapping unit 620 includes a bias curve applying unit 622, apredicted average calculating unit 624, and a stretching unit 626. Themeanings of terms used in the tone mapping unit 620 are shown in Table3.

TABLE 3 Terms Description (x, y) Position information of pixel valuecurrently input. input_val Pixel value currently input. Variance Inputparameter controlling degrees of stretching up and down. asymmetry Inputparameter controlling degrees of stretching up and down to be differentfrom each other. pre_av Predicated average value at current pixel usingneighboring average brightness values

The tone mapping unit 620 performs a local tone mapping operation on thelocal areas after the local brightness average calculating unit 614calculates the local brightness averages, not the entire average of theinput image. In other words, when the brightness value of a pixel isgreater than the brightness values of the neighboring pixels, the tonemapping unit maps the pixel onto a greater value. When the brightnessvalue of a pixel is smaller than the brightness values of theneighboring pixels, the tone mapping unit maps the pixel onto a smallervalue. This is performed to stretch the brightness values of the pixelsup and down in consideration of the states of the local areas. Adifference between the predicted average predicted from the localbrightness average of the local area and the brightness value of thepixel of the input image, the dynamic range information of the inputimage, and variance and asymmetry as input parameters serve asstretching parameters. As the dynamic range is greater and the varianceas the input parameter is greater, the degree of stretching is greaterhere, only when the brightness value of the current pixel is stretchedto a smaller brightness value, the asymmetry is used and is a parameterfor controlling the degree of stretching of the brightness value. Thetone mapping procedure of the tone mapping unit 620 is described now.Here, it is assumed that n is 8.

The bias curve applying unit 622 increases the brightness value using abias curve 910 when the currently input brightness value (input_val) isequal to or less than 128 (see A), and decreases the brightness valueusing the bias curve 910 when the currently input brightness value isgreater than 128 (see B). This can be seen by comparing the bias curve910 with a reference linear function 900 shown in FIG. 9. Comparing thebias curve 910 with the reference linear function 900, the functionvalue of the bias curve 910 is greater than the function value of thereference linear function 900 when it is less than 128. The functionvalue of the bias curve 910 is less than the function value of thereference linear function 900 when it is greater than 128. Here, itshould be understood by those skilled in the art that 128 is a referenceof an embodiment and the reference can be changed.

The predicted average calculating unit 624 predicts the brightness valueof the current pixel position from the local brightness average storedin the average table. When the average of the local area to which thecurrent pixel is included is simply used, it is not possible to avoid ablocking phenomenon of the local areas. Accordingly, the brightnessaverages are predicted using a bi-linear interpolation method.

In FIG. 10, it is assumed that the position of the current pixel 1000 ofwhich the predicted average should be acquired is (x,y). A rectangle1010 in which the current pixel 1000 is located and which is obtained byconnecting the centers of the local areas divided by the localbrightness average calculating unit 1020 is found out. Here, it isassumed that the rectangle 1010 is the rectangle connecting the centers1011, 1012, 1013, and 1014 of the first local area 811, the second localarea 812, the third local area 813, and the fourth local area 814 andthe local brightness averages of the first to fourth local areas 811 to814 are av1 , av2, av3, and av4.

The local brightness averages of the first to fourth local areas 811 to814 are acquired from the average table (mean_table). Distance ratios(x_length, y_length) from the coordinate (x,y) of the current pixel 1000to the side of the rectangle 1010 are calculated using Expression 12.Here, the distance ratios are values expressing the distances to twoclosest sides of four sides of the rectangle 1010 at a ratio of 0 to 1.x_length=((x−1) % mask_(—) s)/mask_(—) sy_length=((y−1) % mask_(—) s)/mask_(—) s   Expression 12

Here, “%” is an operation for calculating a remainder and “/” is anoperation of division.

Two virtual averages av5 and av6 are acquired from the distance ratio byinterpolation using Expression 13.av5=(1−y_length)×av1+y_length×av2av6=(1−y_length)×av3+y_length×av4   Expression 13

A predicted average (pre_av) is predicted from two virtual averagesusing Expression 14.Pre_av=(1−x_length)×av5+x_length×av6   Expression 14

Here, x_length and y_length may be exchanged each other in Expression 13and 14.

Thereafter, the stretching unit 626 acquires a difference between thebrightness value (input_val) of the current pixel and the predictedaverage (pre_av) predicted using Expression 14. When the currentbrightness value is greater than the predicted average(input_val>pre_av), the stretching unit acquires as the final tonemapping result a value obtained by adding the difference (Diff) to abias applied value acquired using the bias curve 910 shown in FIG. 9.When the current brightness value is smaller than the predicted average(input_val<pre_av), the stretching unit acquires the final local tonemapping result a value obtained by subtracting the difference (Diff)from the bias applied value acquired using the biar curve 910 shown inFIG. 9.

The difference (Diff) may be weighted. A difference weighting procedureis shown in FIG. 11. A value obtained by weighting the difference (Diff)as shown in FIG. 11 is added to or subtracted from the bias appliedvalue. The asymmetry and the variance as the input parameters and thedynamic range information are used to control the degree of weighting atthe time of constructing a weighting curve. As the dynamic range isgreater and the variance is greater, the degree of stretching using thedifference (Diff) is greater, only the degree of subtraction (−) may beindependently controlled using the asymmetry parameter.

In this embodiment, the preprocessing unit 610 can acquire the dynamicrange information and the local brightness average from the previousframe in advance and the tone mapping unit 620 can the dynamic rangeinformation and the local brightness averages acquired from the previousframe to the pixels of the current frame. The imaging apparatus 100according to the invention can control the brightness in real time.

A local image brightness control method is described now with referenceto FIG. 12. FIG. 12 is a flow diagram illustrating the local imagebrightness control method.

First, an image is input (step S1210). The input image is color dataincluding brightness values and it is hereinafter assumed that thebrightness value V of the HSV data is controlled. Here, it is assumedthat the brightness values of the input image have an n-bit resolution.

Then, the dynamic range information of the input image is acquired (stepS1211). This has been described with reference to FIG. 7 and thus isdescribed in brief.

A histogram is acquired by scaling the brightness values of the pixelsin the input image to the range of 0 to 2^(n)-1 and expressing thenumber of cases where the corresponding values appear in the inputimage. A cumulative density function where the acquired histogram isaccumulated in the order of brightness values (from 0 to 2^(n)-1) isthen acquired. An image range of the pixels in the input image is set.In order to find out the distribution range of the brightness values ofthe input image, the brightness values of the pixels, in which thevalues obtained by dividing the CDF values of the pixels by the size ofthe input image correspond to the boundary predetermined or input by auser, are set as the global minimum value (global_min) and the globalmaximum value (global_max).

Then, the local brightness averages of the input image are acquired(step S1223). The local brightness average calculating method has beendescribed in detail with reference to FIG. 8 and thus descriptionthereof is omitted.

A bias curve is applied to the pixels of the input image (step S1225).An example of the bias curve is shown in FIG. 9 and it should beunderstood by those skilled in the art that various other bias curvescan be used.

The processes of steps S1221, S1223, and S1225 can be performedconcurrently or with a constant interval.

Posterior to step S1223, a predicted average corresponding to theposition of a current pixel is calculated using the bi-linearinterpolation method from the local brightness averages (step S1230).This has been described in detail with reference to FIG. 10 and thusdetailed description thereof is omitted.

In step S1240, the value (having been subjected to step S1225) obtainedby applying the bias curve to the brightness value of the current pixelof the input image is compared with the predicted average and the inputimage is then stretched using the dynamic range information acquired instep S1221.

Here, when the brightness value of the current pixel is greater thanthose of the neighboring pixels, the brightness value is mapped onto agreater value by the stretching. When the brightness value of thecurrent pixel is less than those of the neighboring pixels, thebrightness value is mapped onto a smaller value. That is, the brightnessvalue of the current pixel is stretched up or down in consideration ofthe states of the local areas, thereby performing the local tonemapping.

In another embodiment of the invention, the processes of steps S1221,S1223, and 1225 may be concurrently performed to the current image instep S1240 and the bias curve applied value of the current frame, thedynamic range information of the previous frame, and the localbrightness information of the previous frame may be used in stretchingof step S1240, thereby controlling the image brightness in real time.

According to the third embodiment of the invention, the image brightnesscontrol device 220 can primarily improve the definition of brightness ofthe entire image or secondarily improve the definition of brightness ofthe local areas using the local brightness information.

Since the primary improvement in definition of brightness of the entireimage has been described with reference to FIGS. 3 to 5, and thesecondary improvement in definition of brightness of the local areasusing the local brightness information has been described with referenceto FIGS. 6 to 12, detailed description thereof is omitted.

FIG. 13 shows an original image and FIG. 14 is a diagram illustrating anoutput image acquired by applying the image brightness control methodaccording to the first embodiment of the invention to the original imageshown in FIG. 13. FIG. 15 shows an original image and FIG. 16 is adiagram illustrating an output image acquired by applying the imagebrightness control method according to the second embodiment of theinvention to the original image shown in FIG. 15. Compared with theoriginal images shown in FIGS. 13 and 15, it can be seen that the outputimages shown in FIGS. 14 and 16 have the improved definition ofbrightness.

On the other hand, the above-mentioned image brightness controllingmethod (see FIGS. 5 or 12) may be provided as a computer program. Codesand code segments of the program can be easily reached by computerprogrammers in the art. The program may be stored in a computer readablerecording medium and may be read and executed by a computer to embodythe image brightness controlling method. The computer-readable mediumincludes a magnetic recording medium, an optical recording medium, and acarrier wave medium.

Although the invention has been described with reference to theexemplary embodiments, it will be understood by those skilled in the artthat the invention can be modified and changed in various forms withoutdeparting from the spirit and scope of the invention described in theappended claims.

As described above, the image brightness controlling device and theimage brightness controlling method according to the invention cancontrol the dynamic range to be applied to an output image depending onthe range of pixel values of an input image and input parameters.

It is possible to improve the definition of brightness so as to bematched with the characteristics of an input image by automaticallyconsidering how the distribution characteristic of an image should bereflected.

It is also possible to acquire the natural quality of image with stillsmaller noise than that of the conventional methods.

It is also possible to improve the definition of brightness bycontrolling the degree of stretching in the unit of local regions of aninput image.

It is also possible to easily acquire the natural quality of image withsmall noise by controlling the input parameters (such as variance valuesand asymmetric parameters) in consideration of the degree of noise of aninput image.

It is also possible to optimize the memory using the minimum memoryspace without requiring the troublesome calculations such as Fouriertransform and several pieces of frame memories.

It is also possible to apply the invention in real time, by performing atone mapping operation in the unit of pixels of a current frame usinginformation acquired from a previous frame.

While the present invention has been described with reference topreferred embodiments, it will be understood that various changes andmodifications may be made by those skilled in the art without departingfrom the spirit and scope of the present invention, as defined by theclaims appended below.

What claimed is:
 1. An image brightness control device to controlbrightness in real time, the device comprising: a preprocessing unitacquiring from a previous frame of an input image, dynamic rangeinformation corresponding to an image range of the input image usingbrightness values of color data of the input image and acquiring localbrightness averages of local areas obtained by dividing the input imageinto the local areas having a predetermined size; and a tone mappingunit performing on a current frame of the input image, a local tonemapping operation on the local areas by comparing a bias applied valueusing a bias curve with a predicted average of a pixel in the inputimage predicted from local brightness values of the local areas, and notan entire brightness average of the input image, wherein thepreprocessing unit includes: a dynamic range information acquiring unitsetting a predetermined region of the brightness values of the pixels ofthe input image as an image range; and a local brightness averagecalculating unit dividing the input image to a plurality of the localareas and then calculating the local brightness averages which areaverages of the brightness values of the pixels in the plurality of thelocal areas, wherein the dynamic range information acquiring unitincludes: a histogram acquiring unit expressing the number of caseswhere the brightness values appear in the pixels of the input image as ahistogram; a cumulative density function (CDF) calculating unitcalculating a cumulative density function obtained by accumulating thehistogram with respect to the brightness values; and an image rangesetting unit setting a predetermined area of the cumulative densityfunction to the image range, and wherein the tone mapping unit includes:a bias curve applying unit acquiring the bias applied value by applyingthe brightness value of the pixel to the bias curve, wherein the biasapplied value is less than the brightness value when the brightnessvalue of the pixel is equal to or greater than a threshold value and isgreater than the brightness value when the brightness value is less thanthe threshold value; a predicted average calculating unit calculatingthe predicted average from the local brightness averages by the use of abi-linear interpolation method; and a stretching unit stretching thebrightness value using a difference between the bias applied value andthe predicted average.
 2. The image brightness control device accordingto claim 1, wherein the tone mapping unit thereby acquires the biasapplied value of a pixel in the input image, calculates the predictedaverage from the local brightness averages, and stretches the brightnessvalue of the pixel.
 3. The image brightness control device according toclaim 1, wherein the stretching unit applies a weight predetermined orinput from a user to the difference between the bias applied value andthe predicted average.
 4. The image brightness control device accordingto claim 3, wherein the weight is set to enhance the degree ofstretching as the dynamic range based on the dynamic range informationis wider.
 5. The image brightness control device according to claim 3,wherein the weight is independently controlled on the basis of anasymmetry parameter input from the user.
 6. An image brightness controlmethod to control brightness in real time, the method comprising:acquiring from a previous frame of an input image, dynamic rangeinformation corresponding to an image range of the input image usingbrightness values of color data of the input image; acquiring from theprevious frame of the input image, local brightness averages of localareas obtained by dividing the input image into local areas having apredetermined size; and performing on a current frame of the inputimage, a local tone mapping operation on the local areas by comparing abias applied value using a bias curve with a predicted average of apixel in the input image predicted from local brightness values of thelocal areas, and not an entire brightness average of the input image,wherein the step of acquiring the dynamic range information includes thesteps of: expressing the number of cases where the brightness valuesappear in the pixels of the input image as a histogram; calculating acumulative density function (CDF) obtained by accumulating the histogramwith respect to the brightness values; and setting a predetermined areaof the cumulative density function to the image range, and wherein thestep of performing the local tone mapping operation includes the stepsof: acquiring the bias applied value by applying the brightness value ofthe pixel to the bias curve, wherein the bias applied value is less thanthe brightness value when the brightness value of the pixel is equal toor greater than a threshold value and is greater than the brightnessvalue when the brightness value is less than the threshold value;calculating the predicted average from the local brightness averages bythe use of a bi-linear interpolation method; and stretching thebrightness value using a difference between the bias applied value andthe predicted average.
 7. The image brightness control method accordingto claim 6, wherein the step of acquiring the local brightness averagesincludes the steps of: dividing the input image to a plurality of localareas; and calculating the local brightness averages which are averagesof the brightness values of the pixels in the plurality of local areas.8. The image brightness control method according to claim 6, wherein thestep of stretching the brightness values includes applying a weightpredetermined or input from a user to the difference between the biasapplied value and the predicted average.
 9. The image brightness controlmethod according to claim 8, wherein the weight is set to enhance thedegree of stretching as the dynamic range based on the dynamic rangeinformation is wider.
 10. The image brightness control method accordingto claim 8, wherein the weight is independently controlled on the basisof an asymmetry parameter input from the user.